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Abstract 

Background: Chronic diseases are major causes of disability worldwide with rising prevalence. Most patients 
suffering from chronic conditions do not always receive optimal care. The Chronic Care Model (CCM) has been 
developed to help general practitioners making quality improvements. The Patient Assessment of Chronic Illness 
Care (PACIC) questionnaire was increasingly used in several countries to appraise the implementation of the CCM 
from the patients' perspective. The objective of this study was to adapt the PACIC questionnaire in the French 
context and to test the validity of this adaptation in a sample of patients with multiple chronic conditions. 

Methods: The PACIC was translated into French language using a forward/backward procedure. The French version 
was validated using a sample of 150 patients treated for obstructive sleep apnea syndrome (OSAS) and having 
multiple chronic co-morbidities. Several forms of validity were analysed: content; face; construct; and internal 
consistency. The construct validity was investigated with an exploratory factorial analysis. 

Results: The French-version of the PACIC consisted in 18 items, after merging two pairs of items due to 
redundancy. The high number of items exhibiting floor/ceiling effects and the non-normality of the ratings 
suggested that a 5-points rating scale was somewhat inappropriate to assess the patients' experience of care. The 
construct validity of the French-PACIC was verified and resulted in a bi-dimensional structure. Overall this structure 
showed a high level of internal consistency. The PACIC score appeared to be significantly related to the age and 
self-reported health of the patients. 

Conclusions: A French-version of the PACIC questionnaire is now available to evaluate the patients' experience of 
care and to monitor the quality improvements realised by the medical structures. This study also pointed out some 
methodological issues about the PACIC questionnaire, related to the format of the rating scale and to the structure 
of the questionnaire. 



Background 

Chronic diseases are major causes of disability worldwide 
and prevalence is rising [1]. In France, the number of 
patients officially identified as having at least one chronic 
condition rose from 3.7 million in 1994 up to 9 million in 
2008, and could reach 12 million in 2015 [2]. In addition 
to the medical burden, the growth of chronic conditions 
places a high financial burden on the healthcare systems. 
In 2007, the amount of public insurance reimbursements 
to those patients was estimated at 80 billion euros [3]. 
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Despite these large expenditures, patients suffering 
from chronic conditions do not always receive optimal 
care [4,5]. There is evidence that a long-term, structured, 
and proactive approach can help to reduce the burden 
of chronic diseases, but the organization of medical care 
is still mainly oriented toward acute conditions [6]. The 
management of chronic conditions is often more com- 
plex than that of acute conditions, because i) it involves 
several actors practicing in different sites, ii) the patient 
plays a key role in the treatment effectiveness through 
his/her health habits and regular compliance, and iii) 
given the long-term nature of the chronic conditions, 
chronic care has to deal with symptoms and to antici- 
pate further complications. Then acute care organization 



© 2014 Krucien et al.; licensee BioMed Central ttd. This is an Open Access article distributed under the terms of the Creative 
BlOlVlGCl C^ntrBl Commons Attribution ticense (http//creativecomn ons.org/lii :nsi which perm ts unrestricted use, distribution, and 

reproduction in any medium, provided the original work is properly credited. 



Krucien et al. BMC Health Services Research 2014, 14:269 
http://www.biomedcentral.com/1472-6963/14/269 



Page 2 of 12 



is to some extent inappropriate to adequately manage 
patients with chronic conditions and to achieve effective 
and high quality healthcare system [7,8]. 

In practice, the management of chronic conditions is 
mainly based on primary care and especially on services 
dispensed by the general practitioner (GP) [9]. In most 
countries, including France, the GP usually acts as a 
gatekeeper and he/she is the first health professional to 
adapt to the growing burden of chronic conditions [10]. 
The chronic care model (CCM) has been developed to 
help GPs make the transition from acute care to chronic 
care [11] and it is widely adopted in several countries as 
a framework to reform the organization of GP care. The 
CCM describes a set of 6 elements designed to optimize 
the management of chronic illness, namely community 
linkages, organizational support, self-management sup- 
port, delivery system design, decision support, and clin- 
ical information systems. The goal of treatment aligned 
with CCM principles is to create an informed, activated 
patient interacting with a prepared, proactive practice 
team, resulting in productive encounters and improved 
outcomes [12]. There is evidence that interventions 
containing at least one CCM element could lead to im- 
proved patient care and health outcomes [13-15]. For 
example, patients suffering from chronic obstructive 
pulmonary disease and benefiting from a care including 
CCM components showed lower rates of hospitalizations, 
and shorter hospital stays [16]. 

In line with the increasing use of the CCM to improve 
the quality of the chronic care and the recent trend to 
make healthcare systems more accountable for the pa- 
tients' needs, Glasgow et al. have developed the "Patient 
Assessment of Chronic Illness Care" (PACIC) question- 
naire to appraise the implementation of the CCM from 
the patients' perspective [17]. To date the PACIC ques- 
tionnaire has been used in several countries to evaluate 
the delivery of CCM activities for a variety of chronic 
conditions (Table 1), but there is no version available in 
French language. In addition, validation studies have 
shown mixed evidence regarding the data quality and 
the properties of the PACIC scales [18,19]. 

The objective of this study was to translate the PACIC 
questionnaire into the French language and to test the 
validity of this adaptation in a sample of patients suffer- 
ing from obstructive sleep apnea syndrome (OSAS) and 
other frequent chronic conditions. We were especially 
interested in the validation of the PACIC survey in a 
sample of patients suffering from multiple chronic con- 
ditions, because in practice GPs face a growing number 
of these patients and most of the time these patients re- 
quire more complex medical care. 

The remainder of this study is divided into four sec- 
tions. Section 1 presents the methods used to translate and 
validate the PACIC questionnaire in the French language. 



Section 2 presents the results of the French-PACIC ques- 
tionnaire and the level of achievement of the CCM recom- 
mendations in the French context. Section 3 discusses the 
results of this study. Finally, section 4 provides concluding 
remarks on this adaptation and validation of the PACIC 
questionnaire in French context. 

Methods 

Sample of patients with multiple chronic conditions 
Definition of the sample size 

The sample size was defined according to Hatcher (1994) 
who recommended that the number of subjects should 
be 5 times the number of variables (equivalently a 5:1 
subject- to-variable (STV) ratio), with a minimum of 
100 subjects [20]. In a recent 2-years literature review 
of the published factorial analyses (exploratory factor 
and principal components analyses), Costello & Osborne 
[21] showed that 40.5% of the studies used a sample size 
with a STV ratio lower than 5:1 and 36.8% of the studies 
with a ratio larger than 10:1. In a simulation study, 
MacCallum et al. (1999) showed a perfect recovery of 
population factor structure with a 3:1 ratio (i.e. 60 sub- 
jects and 20 variables) [22]. In this study, we expected 
a maximum number of 20 items. Then a minimal sample 
size of 100 respondents was required. Finally we decided 
to recruit 150 individuals to obtain a large enough sample 
after accounting for missing values. 

Identification of the patients suffering from multiple 
chronic conditions 

The OSAS chronic condition was used as a way to identify 
patients suffering from multiple chronic conditions already 
analysed in the international literature (e.g. Diabetes). 
As shown in this study, on average the recruited pa- 
tients suffered from 2.2 chronic conditions in addition 
to their OSAS. In addition, at least in the French con- 
text, the GP is usually not involved in the management 
and treatment of the OSAS, which is largely treated by 
hospital-based pulmonary specialist. Then the OSAS 
condition was assumed to be of minor influence (if any) 
on the patients' perception of the quality of the chronic 
care delivered by their GP. 

To rule out a potential influence of the OSAS condi- 
tion on the results and to be included in the validation 
study, patients had to meet three eligibility criteria. First, 
they had to suffer from at least 1 chronic condition in 
addition to their OSAS to be considered as patients with 
multiple chronic conditions. Second, they had to have 
experienced at least 1 GP visit in the last 6 months. This 
criterion was used to ensure that respondents had a 
minimal experience of the GP care. Third, they had to 
comply with their OSAS treatment for at least 1 year. 
Under these conditions the OSAS become asymptomatic 
and the treatment is well accepted by the patients. This 
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Table 1 Literature review of the studies using PACIC questionnaire (N = 22) 



Study 


Country 


Sample 
size 


Chronic 
condition(s)* 


Factor 
analysis 


Gender 
(% male) 


Age 

(mean + SD or %) 


PACIC mean 
score (SD) 


Maindal et al. [37] 


Denmark 


481 


DM 


Confirmatory 


60% 


66,4 (10,7) 


2,83 (-) 


Rosemann et al. [25] 


Germany 


236 


OA 


Exploratory 


45% 


- 


2,44 (1,1) 


Schmittdiel et al. [46] 


USA 


4,108 


RESP; DM; HF; 
CAD; Pain 


No 


50% 


61,9 (14,7) 


2,7 (1,1) 


Vrijhoef et al. [44] 


Netherlands 


89 


COPD; HF; Arthritis; 
Geriatric disorders 


No 


49% 


56% " > 65 years" 


2,6 (-) 


Taggart et al. [42] 


Australia 


2,438 


DM; IHD 


Exploratory 


47% 


63,2 (-) 


3,01 (1,07) 


Taggart et al. [42] 


Australia 


914 


DM; IHD 


Exploratory 


43% 


57,4% " > 60 years" 


3,07 (1,06) 


Aragones et al. [27] 


USA (Spanish language) 


100 


DMi 


Confirmatory 


21% 




3,17 (0,82) 


Wensing et al. [41] 


Netherlands 


165 


DM; COPD 


Exploratory 


53% 


68 (10,3) 


2,9 (1) 


Rick et al. [39] 


UK 


2,439 


Various 


Confirmatory 


51% 


51,7% ">65 years" 


2,4 (0,87) 


Ludt et al. [47] 


Europe 


1,745 


CVD 


No 


68% 


67,8 (9,9) 


2,84 (-) 


Szecsenyi et al. [45] 


Germany 


1,399 


DM 


No 


46% 


70,3 (8,5) 


2,86 to 3,21 
Glasgow 


Glasgow et al. [24] 


USA 


266 


Various 


Confirmatory 


44% 


64,2 (10,5) 


2,6 (1) 


Mackey et al. [26] 


USA 


1,823 


DM 


No 


35% 


51,5 (16,7) 


3,05 (1,19) 


Glasgow et al. [1 7] 


USA 


363 


DM 


No 


53% 


64,1 (11,9) 


3,2 (0,9) 


Cramm & Nieboer [38] 


Netherlands 


1,321 


CVD 


Confirmatory 


53% 


63,8 (10,2) 


2,68 (0,86) 


Cramm & Nieboer [38] 


Netherlands 


917 


COPD 


No 


52% 


63,99 (10,19) 


2,89 (0,89) 


Cramm & Nieboer [48] 


Netherlands 


1,570 


CVD 


No 


53% 


66,11 (10,57) 


2,77 (0,86) 


Cramm & Nieboer [50] 


Netherlands 


892 


COPD 


No 


54% 


66.1 (10.6) 


2.89 (0.9) 


Cramm & Nieboer [49] 


Netherlands 


283 


Older people 
(>65 years) 


Confirmatory 


46% 


75.8 (6.8) 




Cramm & Nieboer [51] 


Netherlands 


548 


COPD 


No 


54% 


69.1 (10.2) 


2.85 (0.92) 


Gugiu et al. [19,28] 


USA 


529 


DM 


Confirmatory/ 
Exploratory 


52.70% 


63,4 (-) 




Spicer et al. [18] 


New Zealand 


307 


DM; CDV; 
RESP; Pain 




56% 


68,4 (-) 




Mcintosh [40] 


Canada 


936 




Confirmatory 









*DM: Diabetes; CVD: Cardiovascular disease; OA: Osteoarthritis; CAD: Coronary artery disease; COPD: Chronic obstructive pulmonary disease; IHD: Ischaemic heart 
disease; HF: Heart failure; RESP: Respiratory problems (e.g. Asthma). 



last criterion was used to ensure a homogeneous group 
of patients in terms of OSAS treatment. 

Recruitment of the patients and administration of the survey 

The respondents were consecutively recruited between 
October 2010 and June 2011 in a hospital setting to 
avoid biases related to recruitment in GP office when 
the purpose of the study was to ask patients about their 
GP care (e.g. responses heavily based on the last consult- 
ation experienced, respondents' inhibition) [23]. The pa- 
tients were explicitly told that the survey was about the 
medical services delivered by their GP to manage their 
chronic conditions. This perspective was facilitated because 
the patients were first asked about their GP consultations 
and their chronic conditions to ensure their eligibility. 
Then when they answered the PACIC questions they were 
explicitly told to refer to the above mentioned chronic 



conditions and GP consultations. The data collection was 
done using a self-completed survey that was administered 
by a trained nurse during a follow-up visit for OSAS. In 
addition to the PACIC items, the survey included questions 
about respondents' individual characteristics. The patients 
completed the survey in the waiting room. The content 
and design of this study were checked by a local ethic com- 
mittee (Comite de Protection des Personnes Ile-de-France 
VI, Pitie Salpetriere, France), which provided agreement for 
the conduct of the study. Written informed consent was 
obtained from the patients for the publication of this report 
and any accompanying images. 

Initial version of the PACIC questionnaire 

The initial version of the PACIC questionnaire was de- 
veloped by Glasgow et al. in the United-States with 283 
patients suffering from various chronic illnesses [17]. 
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This initial version consisted in 20 items divided into 5 pre- 
specified dimensions of the GP care as recommended by 
the CCM (i.e. patient activation; delivery system/practice 
design; goal setting/ tailoring; problem solving/contextual; 
follow-up/coordination). The 5 PACIC subscales did not 
match perfectly with the 6 elements of the CCM because 
some aspects of the model, such as clinical information 
systems and organizational support, are generally not 
visible to patients. These 20 items were chosen from 46 
items designed by national experts on chronic illness care 
and the CCM. 

The achievement level of each of these items was rated 
by patients on a 5-point rating scale with the following 
anchors: Almost never (1), Generally not (2), Sometimes 
(3), Most of the time (4), Almost always (5). However, in 
the literature, studies have imposed marginal changes on 
the format of the initial rating scale. Glasgow et al. [24] 
and Rosemann et al. [25] stated that "Each item was 
scored on a 5-point scale ranging from 1 (no or never) to 
5 (yes or always)". Mackey et al. [26] used a 5-points 
scale ranging from "None of the time" through "Always". 
Aragones et al. [27] used a 5-points scale with the follow- 
ing anchors, which are also those of the current official 
version of the PACIC survey in English language: None of 
the Time, A Little of the Time, Some of the Time, Most of 
the Time and Always. Other studies have imposed changes 
on the rating scale. Gugiu et al. [19,28] employed an 11- 
point percentage scale ranging from 0% to 100% by units 
of 10% with two end points labelled "Never" and "Always". 
In this study, the 5-points rating scale was based on the 
following anchors: Never (1), Rarely (2), Sometimes (3), 
Most of the time (4) and Always (5). 

Translation and adaptation of the questionnaire 
Translation 

According to recommendations from the international 
literature [29], the translation of the PACIC items into 
the French language was performed following a forward/ 
backward multi-steps procedure. First, each author trans- 
lated the list of items into the French language without 
discussing this with the other authors. Second, the lists of 
translated items were compared and disagreements were 
discussed, if necessary a 4 th expert was solicited to reach a 
consensus. Third, the agreed list of translated items was 
presented to an external team of health professionals 
and medical researchers, mainly to assess the contextual 
relevancy of the items and the understandability of their 
translation. Fourth, external researchers were solicited 
to perform a backward translation (from French to English 
language) of the items. This multi-steps process appears 
to be in line with the World Health Organization (WHO) 
recommendations to translate and adapt an English 
instrument in different languages [WHO, 2008]. This 
approach putted emphasis on conceptual rather than 



literal (i.e. word-to-word) translations. For this purpose, 
experts involved in the translation process were required 
to have good knowledge of the French and English lan- 
guages, to be familiar with the field of research and to 
be able to translate the concepts avoiding the use of 
any technical jargon. 

Adaptation 

The French version of the PACIC questionnaire was 
piloted with 10 patients suffering from chronic illnesses by 
using individual interviews. The patients were interviewed 
and were asked to explain their responses (i.e. think aloud 
approach). This pilot study was performed to ensure the 
adequacy of the questionnaire content to the French 
context. At this stage, patients had opportunities to suggest 
new items, to modify the translated items, and to suggest 
deletion of some of them. 

Validation of the questionnaire 

The validity of the French PACIC questionnaire was tested 
following three steps: face validity; construct validity; and 
internal consistency. 

Face validity 

The notion of face validity refers to the extent to which 
a test is subjectively viewed as covering the concept it is 
supposed to measure [30]. In practice, the face validity 
of the PACIC is usually assumed to be verified because 
the PACIC questionnaire is directly derived from the 
CCM components. In this study, this assumption was 
tested by asking a pilot sample of patients with chronic 
illnesses about the relevancy of the PACIC items to de- 
scribe their GP care experience. 

In addition, the face validity was investigated through 
analysis of the data quality using descriptive statistics and 
correlation measures. The descriptive statistic included 
mean, standard deviation (SD), percentage of missing values, 
extent of ceiling and floor effects, and normality measures 
(i.e. Kurtosis, Skewness, and Anderson-Darling statistics). 
The floor and ceiling effects referred to the percentage 
of respondents using the most extreme (upper or lower) 
response categories. In practice, a percentage larger than 
20% was associated with floor/ceiling effects [20]. 

Construct validity 

The construct validity was tested using exploratory factor 
analysis (EFA). Given the ordinal and non-normal nature 
of the data collected using a 5-points rating scale, conven- 
tional methods of EFA which rely on the Pearson correla- 
tions and/or maximum likelihood techniques were deemed 
inappropriate. Actually, Pearson correlation coefficients are 
likely to underestimate the true correlations between the 
PACIC items and maximum likelihood approach is based 
on the unwarranted assumption of data normality. In this 
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study, the factor structure of the CCM was explored using 
the Mplus software allowing both to handle ordered cat- 
egorical variables and to use appropriate factoring method 
(i.e. robust weighted least squares [WLSMV]) [31]. 

The Kaiser's measure of sampling adequacy (K-MSA) 
was used to evaluate suitability of data for factor analysis 
[32]. A value between 0.5 and 1 suggested that the sam- 
ple was appropriated for a factor analysis. Second, the 
factors were selected following a parallel analysis imple- 
mented in SAS using a modified version of the Kabacoff's 
code [33]. This approach is based on the idea that non- 
trivial factors should have eigenvalues greater than 
those derived from randomly generated data with the 
same number of variables and sample size. In this 
study, 1,000 datasets were simulated. Each dataset had the 
same dimension than the actual dataset (i.e. 18 variables/ 
columns and 148 respondents/rows). For each simulated 
dataset, the variables were assumed to be normally distrib- 
uted with a mean = 0 and variance = 1. Each time, the 
complete factorial solution is computed (i.e. the number 
of retained eigenvalues is equal to the maximum number 
of variables in the analysis). Then we obtained 1,000 sets 
of 18 eigenvalues computed on simulated datasets. Finally, 
the 95th percentile of simulated eigenvalues was computed 
and used as a lower bound to identify the number of actual 
eigenvalues to select. Only the actual eigenvalues higher 
than the 95th percentile of the simulated eigenvalues were 
considered for the final analysis. Despite being considered 
as the reference approach to identify the number of factors 
to retain, the applicability of the parallel analysis is some- 
what limited when the data are non-normally distributed. 
In this study, the results from the parallel analysis assuming 
normally distributed data were taken as an acceptable ap- 
proximation of those that would be obtained with a parallel 
analysis relaxing the Normality assumption. 

A Geomin rotation procedure was used to simplify the 
interpretation of the factors. The Geomin rotation is 
an oblique rotation method allowing the factors to be 
correlated. The Geomin procedure produces two types 
of matrix: the 'rotated loadings matrix' which includes 
linear combination of variables that make up the fac- 
tor, and the 'factor structure matrix' which includes 
the correlation coefficients between the variables and 
the factors, thus indicating which items measure the 
factors best. In this study, given the limited number of 
factors (i.e. 2), the factors were interpreted according 
to the factor structure matrix only. 

The quality of the factorial solution was evaluated with 
the root mean squared error of approximation (RMSEA) 
was used. It measures the average amount of misfit be- 
tween the model and the data. This measure is obtained 
by investigating the factorial solution in a confirmatory 
factor analysis which analyses the likelihood of observed 
data given the assumed model/structure. In practice, it is 



usually admitted that a value of the RMSEA lower than 
0.05 indicates a 'good' fit, [0.05-0.08] a 'reasonable' fit, 
[0.08-0.1] a 'mediocre' fit and larger than 0.1 a 'poor' fit [34]. 

Reliability 

The internal consistency of the French version of the 
PACIC was assessed using both Cronbach alpha and 
ordinal alpha coefficients. Theoretically, the Cronbach 
alpha is only appropriate when variables are continuous, 
and it has been shown that Cronbach-a is negatively 
biased when it is used to measure the reliability of ordinal 
variables [21]. However this measure is frequently used in 
practice and leads to valid results despite data that are 
highly skewed. According to the literature a value > 0.7 
might be considered as acceptable [24] . 

More recently, the ordinal alpha coefficient has been 
suggested as a better measure when the assumption of 
normality is violated [35]. In their study based on sim- 
ulated data, Zumbo et al. [35] concluded that "ordinal 
alpha provides consistently suitable estimates of the 
theoretical reliability, regardless of the magnitude of 
the theoretical reliability, the number of scale points, 
and the skewness of the scale point distributions. In 
contrast, Cronbach alpha is in general a negatively biased 
estimate of reliability for ordinal data". 

Implementation of the CCM in the French context 

Once the French-version was validated, the responses 
were used to analyze the implementation of the CCM 
recommendations in GP care as perceived by the patients. 
Currently, there is no indication of the best way to treat 
missing values to compute PACIC score(s). We used a 
conservative approach excluding from the analysis the 
respondents who did not fulfill at least 75% of the items. 
According to Glasgow et al., it was more reliable to focus 
on the overall PACIC score (i.e. over all the items) rather 
than on the scores by dimension [17]. Consequently, we 
constructed a composite score using a confirmatory factor 
analysis on the 18 items responses, and performed an 
analysis of variance (ANOVA) of this overall PACIC 
score using the following patients' characteristics as 
covariates: gender, age, self-reported health, number of 
chronic conditions, and number of GP consultations 
per year. The data management and statistical analyses 
were performed using SAS 9.2 and MPlus software. 

Results 

Sample of patients with multiple chronic conditions 

On average the 150 recruited patients had 2.23 chronic 
conditions (SD:1.04) in addition to their OS AS. The 3 most 
frequent chronic conditions were hypertension (27.7%), 
diabetes (17.5%), and cardiovascular diseases (12.5%). 
Regarding their medical care experience, on average the 
patients had consulted a GP 4.15 times during the last 
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12 months (SD:2.07; Median = 4) and a specialist 3.11 
times (SD:2.36) for a total of 7.26 medical consultations 
per year (SD:3.37; Median = 6). Other individual charac- 
teristics are summarized in Table 2. 

Translation and adaptation of the questionnaire 
Translation 

The forward/backward translation procedure was success- 
fully applied. The comparison of the backward translated 
version to the official PACIC survey in English language 
showed two systematic differences. Instead of using an 
indefinite pronoun (e.g. "one told me") and the past tense 
(e.g. "I was informed") to formulate the questions/items, 

Table 2 Characteristics of the respondents (N = 150) 



Characteristic % 
Number of chronic conditions 

2 25.3 

3 40.6 
4+ 34.1 

Gender 

Male 713 

Female 28.7 
Age 

< 60 years 29.5 
[60-65] years 32.2 
[66-70] years 18.1 

> 70 years 20.2 
Self-reported health 

Excellent 2 

Very good 9.3 

Good 36 

Satisfactory 44 

Bad 8.7 
Same GP since 

< 1 year 5.4 
[1-5] years 16.6 

> 5 years 78 
Last GP consultation 

< 15 days 201 
[15-30] days 24.1 
[1-3] months 37.5 

> 3 months 18.3 
Medical structure of the GP 

GP working alone in the structure 52.7 

GP working with other GPs in the same structure 40.7 

GP working with other GPs and health professionnals in the 5.3 
same structure 



the translated version was based on a GP-oriented 
approach (e.g. "the GP told me") and present tense 
(e.g. "I am informed"). These changes arose because 
during the process of translation and piloting the in- 
definite pronoun approach was deemed too vague to 
assess the quality of the care delivered by the medical 
team. In addition, the translated version focused on 
the GP role because in the French context this health 
professional is the main provider of primary care and 
most of the patients have never experienced something 
else. The comparison of the forward translation in French 
language with another untested French version of the 
PACIC survey revealed no major discrepancy between 
the items [36]. These analyses provide support for the 
validity of our multi-step translation process. 

Adaptation 

The pilot study led to two major changes in the translated 
version of the PACIC. First, the set of 20 items was re- 
duced to one of 18 items because 2 items were perceived 
as redundant by the patients. The items "Given me a 
written list of things I should do to improve my health" 
and "Help me to set specific goals to improve my eating 
or exercise" were merged into one item "Help me to list 
things to do to improve my health". In the same way, 
the items "Contact me after a visit to see how things 
were going" and "Ask me how my visits with other doc- 
tors were going" were also merged into only one item 
"Contact me to know how things were going". Actually 
the respondents were able to detect the differences between 
these closely related items, but according to their medical 
experience, they explained that in practice they had no 
way to distinguish one item from each other. Alternatively, 
these apparently redundant items would have been kept in 
the PACIC survey, but they would have been perfectly 
correlated and this would imply methodological difficul- 
ties in the factorial analysis. 

The respondents had the opportunity to suggest new 
items and this did not led to identification of new items, 
thus suggesting that the initial pool of 20 items was 
comprehensive enough to describe the patients' experi- 
ence of GP care. 

Validation of the questionnaire 
Face validity 

The French-PACIC items seemed relevant to describe 
respondents' GP care experience. Respondents did not face 
any difficulties completing the questionnaire. 

Regarding the extent of missing values, only 4 respon- 
dents out of 150 fulfilled less than 75% of the items and 
were then excluded from the analysis. It remained 146 
respondents who provided 2,324 observations out of 
2,336 possible, a missing value rate of 0.5%. This very 
high rate of completion might be partly attributable to 
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the administration process of the survey and to the setting 
in which the survey took place. This 75% cut-off value was 
initially selected according to minimal requirements for 
the factorial analysis. Without running a formal sensitivity 
analysis, our results are expected to be robust to changes 
in this cut-off value because 91.3% of the respondents 
completely filled the survey (i.e. 0 missing value) and 96% 
completed it with less than 10% of missing values. Then, 
given the sample size of this study, including/excluding 
the respondents with a rate of missing values larger than 
10% is not expected to change the results. 

Looking at the % of 'Never' and 'Always' ratings 
(Table 3), 13 items exhibited potential floor effect (i.e. % 
"Never" > 20%), with 8 items having more than 50% of 
their observations concentrated on this modality. Only 4 
items are susceptible to ceiling effect with more than 20% 
of their observations pertaining to the "Always" modality. 
None of the items suffer from both floor and ceiling 
effects. These results suggested non-normal distributions 
and this was confirmed by the Kurtosis and Skewness 
measures that depicted in most of cases positively skewed 
(i.e. S > 0) and leptokurtic (i.e. K < 0) distributions. 

Construct validity 

First, looking at the K-MSA values (Table 4) only 2 items 
showed a bad level of adequacy for a factor analysis, 
namely "The GP encourages me to go to a specific 



Table 3 Descriptive analysis of the responses to the PACIC questionnaire 



Item 


% missing 
values 


% never 


% always 


Median 


Kurtosis 


Skweness 


1. The GP asks about my ideas when we made a treatment plan 


2.0% 


39.5% 


14.3% 


3 


-1.49 


0.22 


2. The GP helps me to make a treatment plan that 1 could do in my daily life 


4.0% 


34.7% 


1 3.6% 


3 


-1.44 


0.19 


3. The GP gives me a copy of my treatment plan 


2.0% 


83.7% 


2.7% 


1 


6.54 


2.73 


4. The GP gives me choices about treatment to think about 


2.7% 


52.4% 


8.2% 


1 


-0.29 


1.00 


5. The GP thinks about my values and traditions when s/he recommends 
treatments to me 


4.0% 


73.5% 


5.4% 


1 


1.94 


1.81 


6. The GP helps to plan ahead so 1 could take care of my illness even in hard times 


2.0% 


27.2% 


1 1 .6% 


3 


-1.27 


0.14 


7. The GP told me how my visits with other types of doctors help my treatment 


3.3% 


14.3% 


51.7% 


5 


-0.54 


-1.00 


8. The GP shows me how what 1 do to take care of my illness influence my condition 


3.3% 


1 2.2% 


42.9% 


4 


-0.54 


-0.87 


9. The GP asks me to talk about my goals in caring for my illness 


3.3% 


50.3% 


6.1% 


1 


-0.05 


1.07 


10. The GP asks me questions about my health habits 


2.7% 


1 9.0% 


1 8.4% 


3 


-1.06 


-0.32 


1 1. The GP helps me to list things that 1 can do to improve my health 


2.0% 


59.2% 


8.2% 


1 


-0.35 


1.05 


12. The GP refers me to a dietician, health educator, or counsellor 


2.7% 


57.8% 


4.1% 


1 


0.88 


1.35 


13. The GP encourages me to go to a specific group to help me with 
my chronic illnesses 


2.0% 


86.4% 


0.7% 


1 


9.00 


3.00 


14. The GP encourages me to attend programs in the community that could help me 


2.7% 


92.5% 


0.0% 


1 


14.60 


3.88 


15. The GP asks me how my chronic illnesses affect my life 


2.0% 


32.7% 


1 2.2% 


2 


-0.89 


0.60 


16. The GP asks me to talk about any problems with my medicines or their effects 


3.3% 


1 5.6% 


36.1% 


4 


-1.12 


-0.49 


17. The GP asks me if I am satisfied with the organisation of my care 


2.0% 


44.9% 


14.3% 


2 


-1.33 


0.46 


18. The GP contacts me to know how things are going 


2.0% 


1 6.3% 


33.3% 


4 


-0.99 


-0.64 



Table 4 Measure of the sample adequacy (MSA) for 
factor analysis 



Item 


Kaiser MSA 


Group 


1 


0.84 


Very good 


2 


0.85 


Very good 


3 


0.84 


Very good 


4 


0.85 


Very good 


5 


0.73 


Good 


6 


0.85 


Very good 


/ 


0.85 


Very good 


8 


0.82 


Very good 


9 


0.83 


Very good 


10 


0.84 


Very good 


11 


0.78 


Good 


12 


0.8 


Very good 


13 


0.67 


Bad 


14 


0.65 


Bad 


15 


0.9 


Excellent 


16 


0.91 


Excellent 


17 


0.82 


Very good 


18 


0.78 


Good 


Overall 


0.82 


Very good 



Kaiser MSA: > 0.9: Excellent; [0.8-0.9]: Very good; [0.7-0.8]: Good; [0.6-0.7]: Bad; 
[0.5-0.6]: Very bad; < 0.5: Inacceptable. 



GP: General practitioner. 
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group to help me with my chronic illnesses" and "The 
GP encourages me to attend programs in the commu- 
nity that could help me". The overall measure is about 
0.82 and thus suggested that the observations were 
suitable for the factor analysis. 

Second, the parallel analysis led to the identification 
of a factorial solution in 2 factors (Figure 1). It should 
be noted that these 2-factors solutions also satisfied 
the standard Kaiser criterion for the factors identifica- 
tion (i.e. eigenvalues > 1). 

Third, the Geomin rotation procedure led to a substantial 
inter-factors correlation of 0.415, thus suggesting that the 
two dimensions of the PACIC cannot be seen as inde- 
pendent dimension. Looking at the factor structure matrix 
(Table 5), each item was assigned to only one factor with 
the highest contribution (i.e. magnitude of the coefficients). 
According to this approach, the 1 st factor would be best de- 
scribed by 10 items and the 2 nd factor by 8 items. However, 
given the somewhat high level of inter-correlation between 
the two factors, some items {3;4;9;10;11;15} seemed to 
be well accounted for by both factors. The first factor 
referred mainly to items describing efforts made by the 



GP to customise the chronic care according to the pa- 
tients' needs and values (e.g. asking about ideas when 
making the treatment plan; making a treatment plan 
suitable in patients' daily life; taking into account pa- 
tients' values when recommending a treatment; taking 
into account the patients' health habits; making a treat- 
ment plan allowing for changes in patients situation) 
and to items describing the interest of the GP in the 
medical experience of the patients (e.g. asking how the 
chronic conditions affect the patients' life; asking for 
problems with the medicines; asking if patients are sat- 
isfied of the chronic care; contacting patients to know 
if things are doing well). The second factor referred to the 
actions taken by the GP to implement a collaborative 
chronic care with the patients (e.g. giving a copy of the 
treatment plan; giving choice about treatment; providing a 
list of things to do) and other professionals (e.g. referring 
to a dietician; encouraging to go to support group; encour- 
aging to use community/social programs). 

The resulting RMSEA value for this factorial solution 
was 0.083 with a 90% CI=[0.068;0.099], thus suggesting a 
'reasonable' level of fit. 
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Table 5 Results of the factor analysis 





Item 


Factor 1 


Factor 2 


Dimension 




2. The GP helps me to make a treatment plan that I could do in my daily life 


0.707 


0.416 


Customisation 




5. The GP thinks about my values and traditions when s/he recommends treatments to me 


0.498 


0.373 


Customisation 




6. The GP helps to plan ahead so I could take care of my illness even in hard times 


0.730 


0.301 


Customisation 




7. The GP told me how my visits with other types of doctors help my treatment 


0.544 


0.048 


Experience 




8. The GP shows me how what I do to take care of my illness influence my condition 


0.717 


0.208 


Cooperation 


Factor 1 








10. The GP asks me questions about my health habits 


0.649 


0.505 


Customisation 




15. The GP asks me how my chronic illnesses affect my life 


0.710 


0.502 


Experience 




16. The GP asks me to talk about any problems with my medicines or their effects 


0.589 


0.281 


Experience 




1 7. The GP asks me if I am satisfied with the organisation of my care 


0.497 


0.408 


Experience 




18. The GP contacts me to know how things are going 


0.520 


-0.012 


Experience 




1. The GP asks about my ideas when we made a treatment plan 


0.468 


0.573 


Customisation 




3. The GP gives me a copy of my treatment plan 


0.638 


0.673 


Cooperation 




4. The GP gives me choices about treatment to think about 


0.596 


0.684 


Cooperation 




9. The GP asks me to talk about my goals in caring for my illness 


0.551 


0.654 


Customisation 


Factor 2 










1 1 . The GP helps me to list things that I can do to improve my health 


0.605 


0.633 


Cooperation 




12. The GP refers me to a dietician, health educator, or counsellor 


0.384 


0.719 


Cooperation 




13. The GP encourages me to go to a specific group to help me with my chronic illnesses 


0.252 


0.888 


Cooperation 




14. The GP encourages me to attend programs in the community that could help me 


0.318 


0.767 


Cooperation 



Reliability 

Overall, the Cronbach alpha coefficient showed a somewhat 
high level of internal consistency for the 18-items French- 
PACIC questionnaire with a value close to 0.87. This meas- 
ure was also used to test the internal consistency within 
each dimension and the values were respectively 0.81 
and 0.84. The ordinal alpha coefficient led to the same 
conclusions about the reliability of the two dimensions 
with values respectively of 0.89 and 0.88. 

Implementation of the CCM in the French context 

Overall the patients perceived the CCM as being moder- 
ately well implemented in their chronic care (Table 3), 
with a median PACIC score of 2. However this global as- 
sessment hides differences between the two dimensions 
of the questionnaire. The items pertaining to the 2 nd di- 
mension "Collaborative chronic care" appeared to be less 
implemented than those of the 1 st dimension "Personalised 
chronic care", with median scores respectively of 3 and 1. 
However, the Wilcoxon test of signed rank suggested no 
statistically significant difference between the PACIC scores 
of these two dimensions (S = -342.5, P-value = 0.5096). 
These results were consistent with the French context in 
which the GP care is still mainly focused on acute condi- 
tions and therefore most of the CCM recommendations 
were not implemented. The French-version of the PACIC 
can be obtained from the authors on request. 

The ANOVA showed a significant effect of age and 
the self-reported health (Table 6). The patients less than 



65 years old had a mean composite score higher than those 
older than 65years, respectively 2.56 (SD=0.7) and 2.25 
(SD=0.73). Interestingly, the patients who reported a some- 
what good health state (i.e. good, very good, or excellent) 
showed PACIC scores larger than those who reported a de- 
teriorated health state (i.e. satisfactory or bad), respectively 
2.6 (SD=0.73) and 2.3 (SD=0.7). 

Discussion 

This study has successfully translated and adapted the 
PACIC questionnaire to the French context. The psycho- 
metric properties of the French-version have been 
tested and results have provided support of its validity. 
The different analyses have left some issues that are 
discussed below. 



Table 6 ANOVA of the PACIC score according to 
individual characteristics 



Characteristic 


Degrees of 
freedom 


Type III sum 
of squares 


Mean 
square 


P-value 


Number of chronic 
conditions (in class) 


2 


1.00 


0.50 


0.367 


Number of GP 
consultations per 
year (in class) 


2 


0.42 


0.21 


0.657 


Gender 


1 


0.09 


0.09 


0.676 


Self-reported health 


1 


2.43 


2.43 


0.028 


Age (in class) 


3 


1.70 


1.70 


0.019 



Model: N = 147; degrees of freedom used = 9; R2 = 12.2%; P-value = 0.031. 
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Why a structure of the questionnaire in 2 dimensions 
instead of 5? 

The initial English-version of the PACIC consisted in 5 
dimensions derived from the components of the chronic 
care model [17]. These dimensions were pre-specified 
rather than identified through data analysis. In line with 
this initial approach, most of the validation studies have 
used the confirmatory factor analysis (CFA) method to 
verify the adequacy of the data to this 5-dimensions 
structure. Among the 7 studies identified in the litera- 
ture, 4 provided support for a 5-dimensions structure 
[24,27,37,38] and 3 suggested that such structure did 
not adequately fit the PACIC data [28,39,40]. As in our 
study, an alternative approach consisted in using an 
exploratory factor analysis (EFA) to emerge the num- 
ber of dimensions from the data. In the literature this 
approach has been used in 5 studies, with 2 studies 
identifying 5 dimensions [25,41] and the 3 others studies 
identifying 1 or 2 dimensions [28,42]. Our 2-dimension 
structure of the PACIC is more closely related to these last 
validation studies which included items in more general 
dimensions. At the extreme situation, Gugiu et al. discussed 
the potential of PACIC data to emerge only one dimension 
[28]. These differences of structure between the validation 
studies may arise from methodological differences between 
the studies. The studies validating the PACIC survey are 
quite heterogeneous in the way they performed the factor 
analysis (e.g. confirmatory vs exploratory analysis; Pearson 
correlation vs Spearman or Polychorric correlation as input 
matrix; Orthogonal vs Oblique rotation of the factors; 
Kaiser criterion + Scree plot vs Parallel analysis to select 
the factors). As in Gugiu et al. [28], this study relied on 
non-standard, but theoretically more appropriate, meth- 
odological choices (i.e. Oblique rotation; WLS estimator; 
Parallel analysis; Exploratory/Confirmatory analysis) and 
raises doubt about the structure of the PACIC in 5 dimen- 
sions. These factorial differences might also arise from 
true differences between the health systems and the sample 
of patients in which the studies took place. Some studies 
are especially interested on the perspective of patients 
suffering from some chronic condition, whereas other 
studies focused on patients suffering from multiple chronic 
conditions, like our study. 

Why a version of the questionnaire in 1 8 items instead of 20? 

Merging 4 items into 2 was a direct consequence of the 
active role played by respondents in the adaptation of 
the PACIC questionnaire. They merged items perceived 
as redundant. Second, the removal of two items with 
very high level of "Never" responses suggested that these 
items were not appropriate to describe the way chronic 
conditions were managed in primary care in France. As 
previously discussed by Rick et al. (2012), Wensing et al. 
(2008) and Spicer et al. (2010), the PACIC questionnaire 



has been firstly developed in a specific context, namely 
the United States, where the organization of the health- 
care system and current initiatives in terms of chronic 
disease management may differ from other countries 
[18,39,41]. Then in terms of face validity it was deemed 
necessary to adapt the PACIC questionnaire to each 
context in which it was transposed to adequately reflect 
the experience of the patients. Other studies have also 
removed or reframed some initial PACIC items during 
the adaptation/ translation process. Mcintosh (2008) sug- 
gested a 17-items version [40] and Gugiu et al. have de- 
veloped a valid short-form of the PACIC questionnaire 
in 11 items [19]. 

What about the relationships of the PACIC scores with 
individual characteristics? 

In our study, we found significant relationships between 
the overall PACIC score and two patients' characteristics, 
namely the age and the self-reported health. These results 
are partly in line with the literature. In terms of self- 
reported health, 2 studies have previously investigated the 
relationship with the PACIC score(s) and concluded to 
significance [37,42] . In terms of age, the evidence is mixed: 
in the 9 studies which tested the relationship with PACIC 
score, only 3 showed a significant result [38,39]. Regarding 
the other characteristics tested in our study, the results 
are also in line with the literature: 5 studies out of 7 found 
no relationship between number of consultations and 
PACIC score, and 7 studies out of 10 found no relation- 
ship between the gender and PACIC score. Only one 
study has previously analyzed the effect of the number 
of chronic conditions and did not conclude there was a 
significant difference [37], like us. In our study, we have 
not investigated potential differences in PACIC scores 
according to the type of chronic conditions. However, 
Glasgow et al. have tested the initial English-version of 
the PACIC in populations with various chronic condi- 
tions and displayed no differences in its psychometric 
properties across these conditions [17]. Developers of 
the chronic care model argued that this model was 
generic and could be successfully applied to variety of 
chronic conditions [11,43]. 

What about the implementation of the chronic care model? 

According to the overall PACIC score, our results appear 
to be consistent with other studies performed in differ- 
ent countries and chronic conditions. We identified 17 
studies that reported an overall PACIC score which 
ranged from 2.4 [39] to 3.2 [17] with a median of 2.84. 
With an overall PACIC score of 2.6, our study is in line 
with the studies of Vrjihoef et al. and Glasgow et al. re- 
spectively performed on 89 and 266 patients suffering 
from multiple chronic conditions in the Netherlands and 
in USA [24,44] . The comparison of our result with those 
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of Szecsenyi et al. is especially interesting, because 
French and German healthcare systems shared com- 
mon organizational features [45]. In addition Szecsenyi 
et al. used the PACIC questionnaire in two populations 
of diabetes patients, one who participated in a disease 
management programme and the other not. They 
showed a significant difference of overall PACIC score 
between these two groups of patients (i.e. 3.21 vs 2.86). 
A similar disease management programme is currently 
implemented in France and we assume that similar 
pattern of results should be identified using this 
French-version of the PACIC questionnaire. 

Limitations 

It is important to note that our study has some limita- 
tions. First, the primary objective of this study was to 
validate the PACIC survey in the French language on a 
sample of patients suffering from multiple chronic con- 
ditions. For this purpose, we selected patients suffering 
from the OSAS as a well-designed population for this 
study because they are usually suffering from other chronic 
conditions (e.g. diabetes, hypertension, arthritis) and 
they experience complex medical care involving many 
health professionals in different places. Despite this, 
we used selective inclusion criteria to recruit only pa- 
tients with a stabilized OSAS (i.e. asymptomatic condi- 
tion and treatment well accepted by the patients), we 
cannot preclude any potential influence on our results. 
Instead of comparing results across different chronic 
conditions, for future research on the PACIC, it would 
be interesting to compare the ratings after controlling 
for patients' ease of interaction with the healthcare system 
(i.e. number of consultation, self-perceived difficulty to 
get the needed service) and self-perceived health con- 
dition. Second, the sample size of recruited patients 
can be considered as being on the lower bound of ac- 
ceptability. Despite being in line with a part of the 
international literature with a 5:1 subject to variable 
ratio, conventional recommendations are closer to 10:1. 
Moreover, these recommendations are based on continu- 
ous data, which we do not have. The ratio for ordinal data 
is higher. Thus our sample size might be too restrictive for 
further analyses, for example, based on structural equation 
modeling. However our results are still in accordance 
with part of the PACIC literature using same analytical 
approach, thus providing confidence in our results. 

Third, retest reliability was not examined. To date only 
three studies have investigated this issue and showed 
mixed evidence according to intervals (i.e. 2 weeks, 3 or 
8 months) used between the two measurement points 
[17,19,25]. On a sample of 56 patients, Rosemann et al. 
[25] showed a good reproducibility of PACIC scores using 
a 2-weeks interval (r = 0.81). On a sample of 250 patients 
and using a short-version of the PACIC survey, Gugiu 



et al. [19] showed a reasonable test-retest reliability at 
8 months (r = 0.638). 

Fourth, we have not tested the concurrent (or external) 
validity of our adaptation, because we lack of patient sur- 
veys developed and validated in French language. Wensing 
et al. explored this issue using the EUROPEP instrument 
and they showed strong correlations between PACIC and 
EUROPEP scales [41]. 

Fifth, we have not tested the sensitivity to changes of the 
PACIC ratings. Few studies have investigated this issue 
and showed mixed evidence. Maindal et al. investigated 
the sensitivity to change (i.e. enrollment in a disease man- 
agement programme) of the PACIC among 585 patients 
with chronic vascular diseases and showed that the PACIC 
scores improved significantly [37] . 

Conclusion 

In this study, we have successfully translated, culturally 
adapted and validated a French-version of the PACIC 
questionnaire. This tool may now be used in population 
of patients with multiple chronic conditions to measure 
the level of CCM achievement from their perspective, 
and to monitor the quality improvements realized by 
the medical structures, mainly in general practice. 
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